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Introduction 

This is the suimnary of a longer report of the results of a five-year 
evaluation of the Elementary Education Demonstration of the PLATO coraputer-l|asedt>» 
instructional system. The /demonstration was conducted by the Computer-Abased 
Education 'Researqh Laboratory , (CERL) of the University of Illinois, funded by 

the '{Rational Science Foundation. The evaluation was conducted by Educational ^ 

* * 

Testing Service. / . - 

The fu^ evaluation final report and a volume of appendices > including means- 
instruments and preliminary analyses, are available under separate' cover* 



Suinmaxy and Conclusions 
it 

The 1 



The PLATO Elanle^ary Demonstration 



The PLATO Svstem consists of a Xarg§ aentrar computer supporting over 

1,000 widely- distributed and technically sgphisticated terminals. The 
». 

terminals are capablf of delivering print^, animated graphic, color slide, 
aird (with additional feJiuipment) audio information Co students in an Individu- 
^ ^ * ^ a^l^ized manner, "^tuti^? r^ponses entered via an alpha /nujjeric keyboard and a 
touch- sensitive terminal screen, are transmitted to the^ computer and processed 

accdf ding* to ^th*e oroceduras 6,f the oauCicular prograinmed lesson in which the 

^ ' ^ ~ , f ^ ^ ^ - - 

Student*^ is eng^iged* * ^ * I 

The centra^- ♦computer^ return^ feed^»ack to the terminal almast instanta- 

neously^ " Thi^ f eeTlhack^may consist of judgments of the student's response, 

hints, ri^w oroblems^x;<»ntingent on the response, or animated attention directors. 

Extensive ^rcord-keeping facilities make it possible to monitor the performance 

of each student, and to 4^tact aspects of lessons that may need revision. Each 

terminal may also^e used in the authoring mode, to program new lessons for the 

system in the t^TOR language; in thk ins true toi^ mode, to monitor individual 

' performance and to obtain summary information concerning a particular group of 

* students and lessons (a "course"); and as an element in a real-time or in a 

"mailbox" coimnunication network encompassing all terminals. 

With National Science Foundation support, the developers of PLATO^^he Com- 

puter-base'd Education Research Laboratory (QERL) of the Uniyersity of Illinois, 

placed approximately 100 l^LATO terminals and ancillary devices in elementary ^ 

classrooms in the Champaign/Urbana area. Two groups of curriculum developers 

prepared lessons in beginning reading for children from kindergarten to grade 

two and in matheraatics for children in grades four to six. The PLATO lessons 

were designed to supplement, but not to replace, ref^ular classroom instruction. 
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Over the course of the 1975-7'6 demonstration ye^r, over 300 students received 
toathematics lessens via PLATO>*and over 700 students received reading lessons. 

The Evaluation 

This report describes the development, implementation and measured edu- 

cational outcomes of this demonstration project from the point of view of aii 

external evaluator. Although many of the instruments used were summative, or 

"bottom line/* in design* the continuing evaluation of the projects made it 

clear that a more^ormative and process-priented descrf|ption of the context of 

the demonstration was essential for the interpretation of the results o*f the 

1975-76 pilot* and demonstration year periods. Thus, extensive description of 

the experiences and reactions of teachers and students in the classroom as the 

demonstration developed, form the centerpiece of the report. These observa- 

tion and case study reports amply document the fact that the PLATO treatment 

was a collection of treatment^ varying with the classrooms in its acceptance, 

intensity, integration or^ isolation from the ongoing ^ curricula, and interac-- 

tion with teacher coverage and style* 

Because of teacher self-^selection into the PLATO treatments, this was not 

a randomized eJkperiment , but rather a naturalistic study in which comparisons 

could be made, but a multiplicity of plausible explanations could be offered 

for differences in outcomes between PLATO and non-PLATO groups, and among 

, classes itaught by different teachers. We have made these comparisons, and 

have weighed competing explanations |to develop our conclusions* We have also 

attempted to provide sufficient descriptive information to enable others to 

♦ 

weigh alternative explanations where appropriate* 

As this has been written and reviewed, the PLATO system has continued 
to evolve with continuing technical and system developments and further 
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development and refinement of the already effective elementary mathematics 
curriculum. The findings reported here, concerning Impact at the end of the 
1976 school year, may not estimate the effects of the current mathematics 
curriculum. Although revision, of the PLATO Elementary Reading Curriculum 
(PERC) has also taken place, the data reported here do not give evidence that 
techniques for achievement-promoting (Jutcpmes were attained in the reading 
FO?<J^ram during the evaluation period* In the absence of successful experience 
on which io build, it seems as likely that changes could have decreased as 
ificreasera the effectiveness of the reading curriculum. Thus an argument from 
positive trend seems plausible for mathematics achievement, but less so for 
reading. 

Implementation 

As could be expected of an attempt rat simultaneous development and 

manufacture of a technological medium, bf new material for delivery by the 

medium* and of effective ways of using tfiis resource In classrooms, delays and 

modifications in any one component led to dislocations throughout the system* 

In spite of a one year delay in the start of the demonstration, the beginning 

of the pilot year found neither the system nor the lessons ready to deliver 

Instruction sufficiently reliably to warrant summative evaluation. The gener-* 

ally sympathetic reactions of teachers and students to the demands of the 

system and to the efforts of the* developers to improve reliability and effec- 

• * 

tlveness offer insigfits into the resiliance of ^classrooms* They also point 

to the problems inherent in relying on feedback directly from volunteer 

teachers to developers in assessing the complexities and difficulties inherent 

in installing classroom innoyations, or in assessing ti^e effectiveness of such 
* f ♦ 

^innovations. • In the early , stages of implementation, the developers appeared 
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to be working so hard and against such frustration » tl^:^ teachers were, reluc-- 
tant to give anything but positive feedback* / / ^ . ^ . 

As the system stabilized^ and the ^arly ch^os" sAibsided, more teachers 
were willing tq criticize and to d^sqpiprw^re control pver lesson selection and 
4 sequericing /f or their students. 

Daring the pilot year, teacher support consisted partly of meetings^ 
printed Itesson descriptions and various drafts of terminal accessible and hard*- 
copy student progress suiomaries* Som^r* supplementary worksheet material was 

produced in the graph, strand in mathematics* The major mode of ^support* 

/ 

however, was based on classryom visits ^nd interactions with teacherip and 
students by the developers/ either in response to teacher requests for help or 
. to the developers^ desire to obseirve students working *at the terminals and to 
diagnose system and lefsson problems. This teacher support mechanism had the 

drawbacks of expens^^ lack of documentation* for export to new sites, and the 

\ , ^ 

possible fostering of teacher dependence. Given simultaneous development 'and 
implementation, however, this melding of the system witth^ the physical presence 
of developers in the classroom was not surprising • | * ^ 

During the demonstration year, the system stabilized and the mathematics 
curriculum began to appi^oximate its planned form, Mat3i€|Batics developers 
purposely ke^t their classr^lbm presence to a minimum, and although delivery 
of the curriculum did not begin until two months into the schopl year , a 
S reasonable, test of the first-draft curriculum* s effectiveness was possible* 

* • * / /. ■ ■ . ' 

The reading currict^lum, ^ich had appeared to be aV too low a level for"' most 
students sin the pilot year, continued to undergo major revisions at the same 
I time that It expanded its delivery* from 15 to 25 classrooms* The demonstration 

. /yftar saw continued problems with hardware, th^ routing program, and lessons; 
major modifications were introduced in March of the demonstration year* ^. 
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Classroom liaison personnel continued to play a major rple in the reading 
Imp^lementation throughout the demonstration period, < 

9 

Classroom Observations and Itytervievs 
> ♦ 

Information about various aspects of the implementatd^on was obtained from 
teacher interviews and questionnaires and from observations of individual 
students and entire classrooins, • ^ I ^ 

Observations revealed that the physical installation of PLATO te'^;minals 
in *the classroom prov^^d to be relatively trouble-free, althgugh actual opera- 
tion of the terminals had a somewhat disrupting effect initially! and terminal 
fans generallTy raised room noise levels* Frequent System and terminal 
malfunctions added to disrupt iottv^in the pilot year- and audio device ptoblems 
plagued the reading project, well into the demonsti?at4.on yeait* 

a^idents exhibited considerable variation^ with respect to the amount of 
time* needed to develop familiarity ^ith t^minals* The average time needed to 



develop independent* use was about one week, with lower grade Ijevel students 1 

needing considerably more time than upper level students* ^ 

The necessary student * orientation was accomplished by C^RL staff, 

# * / 

teacher aides, and in 'some cases, by* other students who had had previous^ ^ 
experience with the system* Studenli use of PLATO took place primarily within 
school hours, with equal time usually being given t;^ all s^dents* However, 
some students used terminals before and after school, and others gained access 
to terminals at the University on weekends. The volunteer teachers using the 
systeui exhibited considerable heterogeneity with respect to their knowledge 
and expectations '.of , as well as their interest in, the system* Although all 
teachers'' ultimately acquired at legist minimal proficiency with terminals, a 
significant number felt inadequately prepared to deal with terminal probl^s* . 



Teachers' iftjiliarity with the format, style, and eContent of PLATO 
lessons also. varied considerably. Furthermore, teachers showed different 
preferences for various ways of becoming acquainted with lesfons^ with some 
preferring working through lessons themselves and ''others opting to read the 
progfam description book or observe the lessons as their students worked cto 
them, ^ ' ' 

PLATO activities tended to remain largely independent of teachers* regu-- 
Tar curricula, although^ the introducti^ of lesson- prescriptions in the " 
demonstration vear had^ the effe^^^f inflteasi^g integration 'somewhat. 

Generally, teachers* perceptions^ qf PLATO lesson materials wer^ as \ . 

diverse as the lessons themselves and usually depended on teachers* personal 
preferences, the curriculum materials they were currently using, and the 
ability ^<f reactions of their students. 

Although teachers *were divergent in their views on using PLATO materials, 

r • * ^ ■ 

most of them considered PLATO to be an effective means of providing reinforce-- * 
ment, practice, and individualiza^:ion. The attribution of specific effects on 
learning, however > was more difficult. Reading teachers jnost of ten ^mentioned 
increases in the ability to follow directions and^the development of listening 
skills as specific effect of PLATO. Although few teachers mentioned ^ncreased 
faiowledge as an effect, several did report more positive student attitudes 
about subject matter. '-^ ^ 

Generally, teachers believed that the most able students benefited most 
from PLATO, largely because they found it easier to miaster the new method of 
learning offered by PLATO. . 

Math teachers vfere more favorably disposed toward PLATO than were reading 
teachers. Generally, teachers* approval or acceptance of PLATO was conditional, 
with many teachers mentioning the ne^d for system and lesson improvement and 
more adequate implementation. 



- The classrooi observations made during the. pilot and demonstration years 
suggested that the reading classrooms in which PLATO was implemented vere not , 
atypical in their functional use of space and their use of various material 
resourc«i and reading activities, although large-group instruction may have 
been rarer than the norm in these classes. • 

In general, reading teachers exhibited' a low-to-moderat§ level of inter- 
action with PLATO. There was generally little teacher use of the system to 
Obtain information about student perf onflance or to change childrens' PLATO 
assignments. 

Extensive observations of a limited number of k-ind4rs?arten classes during 
the demonstration year showed some diversity among the classroonfs with respect 
to their organization and activity. As might be expected, these kindergarten 
classrooms were quite > different from the first::^d second-grade classrooms 
observed in the pilot year, especially with ihs^S^ to their rare use of text-- 
•books and workbooks. As wi^h first- and- second-grade classrooms, however, 
teachers seldom used the terminal to get information about students' work. 
Students in these classrooms were more often expected to follo^ the PLATO 
schedule, with other classroom activities interrupted when students* turns 
came up, than had been the case in the first- and 'Second-grade classes observed 
in the pilot year.. 

^The mathematics classrooms observed during the pilot year also varied with" 
respect to physical setting and use of material resources and activities. Gen- 
erally, \he predominant feature of these classrooms was their extensive use of 
drill and practice of^maith operations and rules . Mathematics peachers showed 
only a modest amount A>f interaction with 'PLATO, seldom using it to obtain 
additional information about st^den«; or to assign them to'pLATO lessons. Gen- 
erally, mathematics teachers encouraged the use of PLATO, but again, as -a 
Classroom resource, PLATO was .observed to be more isolated than integrated. 

10 ' 



Results of observations of math classrooms during the demonstration year . 
were quite similar^ to those obtained during the previous pilot year. Th.ere ^ 
was an extremely wide variation among classes with respect to top^c coverage ^ 
and use of various materials. Regarding interaction with PLATO, there was^ 
some indicai.t ion that: PLATO scheduling was more strictly adhered to (especially 
in those' classes participating for the.fia;.st tiine) , with PLATCt^aking prece- • 
dence over other activities. Teachers, however, still^^^latively little 
int'ekction with teriflinals and 'tKere continued to bi^ituT effort devoted to 
• relating PLATO content to regular lesions and little' dSi? spent discussing 
PLATO with sttidentsl Despite this relative lack of involvement, howevejp, 
teachers more often made favorable than critical comments about PLATO' s effect/, 
on student learning. Likewise, students were much more likely to find PLATO 
to be enjoyable than frustrating or boring as evidenceji by their comments and 
cheir willingness, to devote uon-allotted or unscheduled tiie to- PLATO. 

Kindergarten and first-grade students' experiences with the PLATO system 
were recorded' through structured observations anti flarr^itive accounts of indi- 
vidual students. These stu^J^nts exhiljited various degrees of mastery of the 
mechanics of the PLATO termW.' Generally, most students used the keyboard . 
effectively and most also showed sufficient facility with the audio unit. 
This was especially tTue with the improved second-gener^tiott ai^dio devites. 
although the time needed to change discs was sometimes longer than desired. 
Microfiche, which was used relatively infrequently by students, was reported 
fco be difficult to use. System (i.e.. software) failures were quite rar*. ' 

while hardware failures were somewhat more frequent and both types were more 

, '''' - 
(indeed, unaccept ably) often noted for reading than for math lessons, ^ 

Generally, students understanding of system direct ji>ns was^rated quite 

jgood. especially for upper grade level students. Their understanding of 

content, though . somewhat' lower . was also rated'good. Most lesson? were judged 

U 



e 
c 



ate difficult/, though a somewhat higher pr6portion of kinder^ 
:-rgrade lessons ware rated to be too easy b^^ observers • Student 
attentiveness to leSsons cldselx paralleled lesson appropriateness* 

Student involvement with PLATO was high at all grade levels, with older 
students showing somewhat more attentiveness* J^reover, most students seemed 
relaxed and confident in their approach to PLATO. . Some verbalizati\in took the^ 
form of requests for help ^ although the number of requests for teacher assis- 
tance was*^ relatively small. Older children were more likely to approach other 
children, instead of the, teacher for help. Childrexi seemed able to help each 
other by guiding or givirtg information rather than by simply giving answers. 

PLATO terminals also served as a social setting for students at^upper 

^ ^ ' ^ "~ ~ ~ . 

grade levels\ Much of the socializing observed .was <;oncemed with She content 

of PLATO* lessons and with helping other students, ♦ Student interactiotj* with - 

CERL staff occurred through frequettt use of the online *'notes** .option < which 

added consideraDle personalization to the PLATO experience. 

A math coverage qi^stionnaire^ showed considerable variation with^ re^^fect 

to curriculutu emphasis among PLATO and comparis^^ teachers^ and ti^e and 

coverage within PLATO strands* PLATO teachers' emphasis In their regtolar 

teaching did tiot necessarily parallel their PLATO emphases, so that there was 

as frequently an inverse as a direct relationship between the emphasis assoc^^r 

ated with on-line and off--line lessons, ^ Furthermore the observed yaria£ ions ^ 

in PLATO topic coverage were judged so extreme that .PLAT<^5Biould not he consi^-- 

ered to have been a single, unifotTn treatment, but was used in different ways 

♦ 

0 either supplement or supplant quite different curricula in different ♦ 
curricula in different classrooms* " * 
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Cise Studies 



^ * 

Six case studies, based on olfservat^ons , l^nterviewsV and teacher logs^^ 
document the problems Of implementation > but also capture teachers'^ asdessiment 
of less, tangible posi^;!^^ and negative* outcomes of opening their classrodms' to 
a st:ill-developin% JtecriKolag^, outcomes not easily open Id^heNaofe obj^tivci 
instruments employe4 ic%^ther comp^ents of « the evaluation/ * " 



Case S.tudy I : describes the experiences of the f irst-gra<Je 
teacher w^s mpst heavily involved Vith the PLATO Elementary , ^ 
Reading CuOTiculUm. Her -approach to reading instruction was ton*- 
sistent with the notion^ of ^ a hierarchy of discrete subskills on 
which the c^rriculimr was initially based* Partly t^eckuse of this * " 
philosophicar compat ability, her expectations were high, and her 
record of er^counters, first 'with the unreliability of the system 
and of the data it yielded,' but increasingly with 'the lessons and 
/^their . sequence," is one of ♦growing frus^ratiot/ • \ 

At the end of the pilot year, her sutamary was, "PLATO needs to 
give better resulCsA* to warrant the time and expei^ cj^ the 
program*'* 

During the demonstration year, her stance t-oward the. curricu^ 
lum became more critical* It was only with the opening up of 
teacher prescription in March that she began to feel that the system 

-i 

w4s beginning to justify the effort it required. Even then, the 
lack "of sufficiently challenging lessons led her to conclude that 
PLATO had not realized the potential she saw for it. She did^ • 

T > . . t 

however, note positive outcomes in motivation to read and w^ite, in 
learning t^type, in feeling of control over a complex machine. 
Toward the ettd of tlj^e year^ sjie allowed some children to switch to 



the.Tpathematies cuisrictilum and reported » "They love it*" 

Learning outcoiijes from th* reading lessons, however, were* seen 

as occurlng' in such areas as fdllowing .diiections and taking turns, 

- i ■ * 

rathef thati in more .specifically reading-related domains. . 

Case Study "11 ; Iliis first-grade teaclfer adopted a less cj>i- 
tical stance to the specific "v^^ of learning activities, fLATO 
.included. Children's enjoyment of |^an' activity was a more bent^. 

feature In her evaluation. She was less involved with the PLATO 

• . \ 

'curricurlum- than was. the teacher in Case Study I, but encouraged 
high usage of the terminals. Although complaining of interrup- . 
fions^ increased time demaivds for scheduling » find/overly easy * 
lessons which did not mesh with^her curriculutn, she retained a ^ 
good-hunfored optimism the end of the pilot year> feeling that 

''most children" enjoyed it and learned many things," but that; 

* . * »' ^ ^ ^ « " 

"There is no way to be specific or li^t the many hidden things 

*' * - 

learned*" As in the previously discussed clas^ learning outcomes 

were sufficiently weJil lyldden to yield n^fejiative ^esults on the 

Metropolitan Achievement Test* 

In the demonstration year, considerable help (four to 'six 

weeks) was required from PERG staff in orienting children to the 

terminal* This teacher began » to question the repetitiveness and 

heavy phonics orient^ation of the lessons. When th^ opportunity 

to prescribe lessons l:ame , she did not become strongly Involved 

in this act i^vity, in keeping with her preference for personal y 

contact over detdiled curri^culum analysis and planning* At the 

and of the demonstration year, in which her students had again 

logged a large number of hours on PLAC^, this teacher was*' 



questioning— ''What has PLATO dope to ma^ a cEange— I don't ka 
if there is that much difference," ^ , ' 



Case' Study III: This mixed K-1 room in one of the two tradi- 
»^ f 

tionally "innovative" schools in the University area, had #1 very 

(iifferent appearance f roiq, the mo^e structured classrooms of the . 

.previous two case studies. Whole-group instruction was rare, with 

individual assignments and, "a steady flow of children in ^d.out of 

small groups, -the environment was complex, individualized, knd 

occasionally, on the surfaie, disorganized. This teacher emphasized 

comprehension over the^ acquisition of discrete subskills, an approach 

not consonant with that of the preponderante of the PM.TO Elementary 

Reading Curriculum (PERC) lessons. Many of the childi\en tn this 

class were already reading at the beginning of the school year. 

Use of the terminals varied with children's interest, a lassaiz-- 

faire approach to scheduling which made PERC staff uncomfortable. 

The teacher, in keeping with her use of multiple routes to %ultipre 

goals, treated PLATO as another resource which might be of value to 

some children. Overall usage in the pilot year was average, but 

with far more individual variation than in most classrooms* 

^t the end of the year, her class, as with most other PLATO ^ 

classes, was lover than non-PtATO classes on the Word Knowledge and 

Wordr Analysis achievement tests, but above comparison classes on the 

Reading Test, a measure of .comprehension least related to the content 

of^ the PLATO Eleajentary Reading Curriculum. 

■ * * 4 

This teacher's enthusiasm for PLATO was low at .the beginning of 

the demonstration year, partly because of the low difficulty level 

of the curriculum, and, partly because of a feeling that the time 
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demwids .fdr proper supervision qf children at the terminals were 
excessive. '^LATD, for the little ones, is not as self-explanatory, 
not as self -correct ingV as they. (PIRC) would like it to be." 
However, when the change to teacher prescription came, and with it,' 
th^pbrtunity to emphdsize sentence and story-level materials, her 
involNfement and enthusiasm increased sharpl3< as 'did, that of her 
pupils. However, this, new level of use a§ain underscored the exces- 
sive time' demands if teacher control oveif lesson assignment was to 
be exerted effectively. 'At the end of the demonstration year, the / 
teacher saw PLAtO, when under her control, to"be useful for manV of 
her 'children, but felt she did not. have time to monitor each-child^s 
progress and prescribe weekly, as she felt she should. - 
Thi^ case.:illul6rates the infeifential value of a single case in 
providing a counterexample' to a hypothesis.' The senior author had * 
approached early interview and classroom observation issues with' tAe 
'assumption .that, PLATO w6uld^Vequire an "open, classroom" en^^i^/onment - 
to gain acceptance, ^nd that more traditional teachers, holding the - 
'vieW that all children should go ^oTigh similar experiences. po§si- 
•bly diffefing only in pace, would resent the interruptions and lack 
•of control resulting from scaring responaiibility with a stand-alone 
V system. This case demonstrates' that a teacher at* th^ more •♦open" 
/end of the continuum, although perhaps lesi concerned with control 
of student movement or form of experiences, was at least as concenied 

♦ 

as were her more "structured" colleagues with the content of these 
experiences and, hence, even more- demanding of control over PLATO. 
Thus the open/ttadltiotial dichotomy did not turn out to be useful 
in Interpreting teacher acceptance, mode or amount of use of the * 
i?tesource. 
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. f Case Study Thi^ sixth-grade^ teacher, with a ^rc^p of^i: 

ab^ity ^nathe^tics kudents, had 26 years of .teaching experien^ 
including experience \ith an earlier mathematics cuns^culum de^^l- 
o^ienti W Wctooiogy effort. She held high expectations for her 
students an<^ for t^LATO.' In what was probably thfe most formarand - 
ildactic' of f the 4lassro<Ws in the demonstration,, the children, as 
-iieM as the'teacl^r, kep^detailed logs 1>f .PLATO lessons, of. 
Iprobrems encountered, and assessments c>f what had been learned. 
*The teacher exp^esM considerable concern over ^rork missed in other 
areas-during their \pLATO time;8lots, which, except for the graphing 
lessoT^s at the beginning of the year, were seen as review of 
'material her studentj had already mastered. She quesfelonM the . 
educational soundness of within-clas^oom terminal placement, as 
opposed to a terminai" room. She registered concern ov*r students' 
attempts, to skip ovei the more difficult exercises, feeling i;hat . 
in some instane^s, "it 'might have fostered carelessness because 
it was easier to punch until correct than think out." iKe theme 
of concern for wasted' time runs through her obserykttons. 

The teacher 'was posit Ae about the general quality of the 
PLATO lessons but was • disappointed at their low level and lack of. 
relationship to her curriculum, and at the eM:ly iSSk^ of teacher 
input in 'the development of 'the strands or control .and prescrip- 

tion of their use. V* * 

'At the end of the year, about one third of her students 
concurred th^t PLATO had not helped them In math.fecaugte it was . 
too easy, but others felt that they had benefited in speed and . 
in understanding, of fractions ai^d" of signed numbers. The teacher, 
■ realizing that her incoming sixth graders in the demonstration 

17 ' - 




year would all have been exposed to PLATO in grade five, and seeing . 
little evidence' of the ^^tem*, flexibility or new. lesson development 
that would be required. to enable her to choose and to integrate 
PLATO lessons with h^r teaching, withdrew from participation at the 
end/of the pilot year.. Although she remained positive about. the. 
pc^exitial of PLATO, the perceived lack of ' cooperative development 
,by teachers and members of the curriculum team convinced her Wt 
the program was not appropriate ^d would iiot become ^appropriate 
for her. students. * , ^ 

Case Study V : TlHs fifth-grade classroom, . in which students^ 

from two classes were grouped for PLATO mathematics and for science 

ins|Tuctioh, was taught by an experienced teacher whose' first love 

was science instruction, with an emphasis on first-hand observation* 

The room and the student??' day were highly structured, but charac- 

terized by a cooperative -and friendly atmosphere • . She was sympatheti 

to an approach to mathematics ^emphasizing problem-solving with under- 

♦.I ~ ~ ^ 

standing — goals highly consonant with th6 emphasis of the PLATO 

niathem§tics curriculum. She tried to encouirage children to think 

and talk about vthe process solving problems, rather than focus- 

r ■ . 

sing only on one procedure and one answer, Howevetr, most mathematics 
instructionar time is still devoted to drill in basic skills.^. 
Indlv^^<iH|il assignments' in*^ varying texts and worksheets, pairing for 
ctfeckdjjifg work,* and iifidiVidual c<msultation with the teacher were 
prominent in the full hox^ per day devoted to largely ihdependent 
work on mathematics assignments • 

* In contrast to the reaction^ of the teacher in Case St^dy IV, 
this teacher found thTat the graphing lessons with which the p'ilot 

. . ' ■ -■ N 
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year began were too difficult for many of her fifth graders. The 
fretjuent requests for assistance with content as veil as mechanical 
difficulties and the need to monitor progress, demanded mori^ time, • 
in class after hours, than she could' give • * j 

With the advent t>f impr<^^d student' record formats frort, the 
fraction strand, this teacher was able to exert more instructional^ 



control, consistent with her technique of allowing chil<ft'en on the 
terminals during only three hours of the day> and of attempting to 
utiliz^^^LATO primarily as drill and practice, reinforcing* topics 
already covered in class. ^ \ , 

In spite of the tight scheduling, usage 'wis high, and test 
results suggested that additional mathematics leaming. Indeed had , 
taken place. The demonstration year went smoothly for this teacher, 
again with positive achievement and attitudinal outcomes. ^ ^ 

This teacher was disappointed to find that children with poor 

work habits carried them over to PLATO, but felt that the tremen- 

* - ~ ,^ 

dous amount of practice provided to middle-achi^ers could only 

benefit these students. Her final c omznen t * about PLATO was that it 

was *Must like any other method — only as good as the people who use 

it/* In this case, the people who used it for curriculum develop- 

ment who had targeted most lessons at an appropriate level for 

these children, and their teacher, who used it as a controlled ^ 

supplement to her curricular decisions, were apparently good enough 



to yield alpositive t^sult in mathematics learning* 




Case Study VI : This fourth^grade class J.s the subiect of a 
separately-'publlshed and highly detailed case stifcy prepared by 
Beiiiadi.ne Evans Stake. This exceptionally talented teacher was 



iatensely involved with PLATO, and achieved striking J)ositiye student 
results in' mathematics* achievement and in attitude toward mathematics* 
gi^It is not r-easonable ,to 'attribute the outcomes in this classroom" 
to PLATO or to the teacheT alone » but it is difficult to separate the 
* influence of the two partners • , In the^case of the graph Strand in** 
the demonstration yjaar, for ^ample, this teacher^ did rtot use most 



PLATO lessons^ judging them to be tpo difficult for her fourth 
gradfers. but, prepared worksheets based on PLATO lessons, with evident 
positive effect* It i;^ not >:le^ in this case\that the teacher would 
,have covered this*toi^ic liad it been for the PLATO experience*^ Thus^ 
we do not attempt to sxjmmarize this case, nor to generalize from it, 
but offer the complete^ study as a?* detailed description of a particu- 
rly successful example of PLATO implementation* * 




D^onstration Year Achievement Outcomes--Mathematics 

Treatment effects^ were estljnated as the difference between observed 
posttest scores and the scores attained by tompariaon of chi^ren with slmila 
valuer of covaridtes (pretest, school, grade> sex, and their interactions) • ' 
PLATO CQverage^reported teacher emphasis, and'strudent charactet;istics were 
taken intc|^ account in interpreting these results. Significant 'average treat- 
ment effects were found for the following grades and instruments: 
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• Grade 4 CTBS Level 2 Computation Subtest . Mi 77 poi .p<,0011 

• ^Curriculum-referenced test Whole +2.79 points p<*01 

Numbers* 

• Curriculum-referenced test Fractions +5.36 points p<.0001 



I " • •' (cont'd.) 
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♦ Gra<^ 5 CTBS »Level 3 Computation Subtest +3.42 lilts ( p<.05 

rf' CTBS Level 2 App.lications. Subtest* +1.21 points 'pf.05 

^ • Curriculum-referenced t^st Ftaqtlons +3,21 points > p<.01 

. • "^"^rriculumrreferenced test Graphs +2.34 poiht^ 

9^ Grade 6 • CTBS Level '2 Computation Subtest - +1.61 points 

• CTBS Level 3 CoRtputation Sulftest +2.87 

♦ Curriculum-ref erended test Fractions 
* • Curriculum-referenced test^ Graphs 




Thus, there were signifiaant positive PLATO effects at all grades on a 
nationally standardized (48-ltem) test of Computation and on, a specially 



constructed (2^ -•item) test of understanding and representation of fractions, 
the two higher grades showed ^p^ificant positive PLATO effects on a test of 
graphs and linear equations, and grade 4 children exhibitjed a significant 
l^ositive treatment effect on a test of understanding of whole ntonber concepts 
and operations* Sach grade-*by-'treatment interaction is consistent with the 
level^pf* the strands: the whole number material representing review for many 
^^^N-ifth and si3cth ^aders, and the graphs* material being quite advanced for many 
fourth graders. 

In addition to these significant overall treatment main effects, signify 
icant school-by- treatment interactions were encounte;red on standardized test 
results in grades five and six, and on curriculum--ref erehced tests at all 
gjrades. These interactions, ^^^^pJip'^Sugges ted that the apparent treatment 
effects a\ross different pairs of teachers varied beyond chance limits, 
were interpreted in the light 'of information available from process data* In 
grade four, a discrepantly high PLATO effect on graphs in one scfiool V class- 
room was attributed to intensive teacher coverage rather than to direct PlATO 
effects* In grade five, a discrepantly low PLATO effect in schooJ-^II-I iev^;2 
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* , • * 

\ > computation an^ whole ritju^bers was interpreted resulting from the teacher's 
dtecision to greatly reduce his own and PLATO 's cQver^ge of these topics on 
' ' the erroneous assumption thal^ they had already been mastered. PLATO fifth ^ 

and sixth graders ^in a mixed grade class in the lowest-achiavlng school did*^ > 
poorly on CTBS Concepts and on whole numbers » reflecting student problems in 
* reading th^ lesson instructions i and support^^g the ^conclpsion, based on pos- 
^icive >s^eatment-by-pretest interaction^, that there^was a general tenderf^ for 
<S - the PLATO exp^riencfe to most benefit xhe initially i*ore able students. 

Even in the^ face of these interactions, estimated PLATO effects were 
positive for all classes in Fractionsji for all fourth-grade classes In !^ole 
Numbers, and for all , fifth- and sixth-grade classes in Graphs^ 
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An apparent treatment --by-sex interaction in gtade six was interpreted as 
an artifact due %d the large number of sixth grade female PLATO students scor- 
ing nlar the filing of the CTBS Level 2 posttest. 

Examination of reported teacher curriculum coverage showed that treatment 
effects werje- greatest on topics which both teacher anS PLATO emphasized, * but - 
that for the majority of topics in which the PLATO teacher reported less 
coverage than did her comparispn counterpart, tteatment effects were still 
po^tive. Particularly in these latter cases, it seems reasonable to attri- 
bute the additional learning to PLATOV, " * " 



'Attitude Outcomes — Mathematics * ^ 



Scales of attitudes toward reading and mathematics yielded: 
# 

# Significant improvements in attitudes toward reading and math-- 
ematics in grade four, with the change being greater in attitude 



toward mathematics^. 
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• An almost significaht (t - 1.80) improvement in attitude / 
toward matiiematics contrasting with a significant decrement in 
attitude tows^d reMing in grade five. - * 

• No measurable impact on attitudes iir^ grade six. 

• A less reliable locus of contr^ol scale did not y^eld signifi- 
cant effects at any grkde, although there was an aJjnost 
s;tgnificant (t * 1,93) tendency for fifth-grade students to ^ . 

. iseeome more external in their attributions of responsibility 
for academic success. The nonsignificant effects at the other 

* 

two grades were in the direction of increased intemality, 
• however. . . - 

Although positive, these attitude scale resul^^ are not ^s dramatic a^-are 
the mathematics achievement results, nor are they as large as those obtained 
during the pilot year, However, examination of individual item responses 
suggests that attitude toward ma themkics as a whole may not be as useful a 
construct as are attitudes toward fractions, decimals, graphs, the latte^ of'* 
which decline-d in favorableness, in the face of sharply rising popularity of 
fractions and decimals among PLATO students. 

Items concerning PLATO itself revealed great majori%s at each grade 
agreeing 'that "PLATO Is fun" and "helps me like math better." and majorities 
of fourth and fifth graders (but 49% of sixth graders)' asserting that "1 learn 
math more easily on PLATO." However, almost one-third of PLATO fourth and ' 
fifth graders and over half of sixtfx gradeVs also agreed that "PLATO is fun at 

. Teachers' -.^bservarions aM evalua- ■ 

tor's. interviews suggested that a primary source of diJIfection among students 
was a lack of more advanced lessons for children who had completed the existing . 
curriculum. 
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Thelnajo^ity of students at all grades agreed that "There are a lot of 

and further that get mad jw^beoi PLATO doesn't 
workJ* • , 

•Although fifth graders assented to "1 like math better with PLATO than 

with my teacher,'* by more than two to oneKXtht other two^ grades being about 

*♦ ' ** 

equally divided) , children at all three grades disagreed with "I learn more 

* * / 

math from PLATO thtfn from my teacher ,''-*y well over two* to one, suggesting that 
many children clearly differentiated "enjoyment from? learning, and saw PLATO as 
more strongly related to the former, A clue to one source of* this different!^ 

ation is in curriculum integration, rather than in the hardware. Approxxmately 

(, • ' 

one-third of PLATO students at each grade level agreed that "It's* hard to see 

how PLATO math lessons fit together/' a considerably l^ger proportion than 

agreed with this description of their teachers^ lessons, 

* 

. Demonstration Year Reading Outcomes 

Significant negative pilot-^eAr results inr'a well-controlled study of . 

grade one PIlATO Elementary Reading Curriculum outcomes led to a reassessment 

of evaluation priorities for the demonstration year. The readiness-oriented 

curriculum and malfunctioning automated management system held first graders 

f * ' 

in lettoer^-recognition and phonics long after the point at which these skills 
had been mastered off-line and children K^sre reading* 

The reading developers placed their highest demonstration year priority 
on Improving introductory materials, rather than on extending coverage tb 
blending^ and comprehension content appropr|.ate to the end of ^ grade one. There- 
fore, it was decided by the National Science Foundation, a congeries of 
consultants, and the evaluator, that evaluation of the first-grade reading 
program during The demonstration year should focus on the process data 
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derived from observations and interviews, wi^h^ resources that had^been allocated 
to securing grade bn6 control 'groups diverted to strengthening the matheaatics 
■evaluation. As'the case stuclie)^ and* observations revealed, although most 
teachers maintained a positive view o^ the pbtential of iPLATO for teSfching 

^ 

readings and felt that procedural. lean\ings had taken place, few specific ^ 
reading outcomes from this curriculum wer^ noted. It should be kfcpt in mind . 
that these teachers had a considerable investment of time- and effort in the 

\ ■ ■ 

trial of the: system, and were to be expected to see the results of PLATO in a 

» 

rather pos4.tive light/ • ^ . 

It was possible to conduct a controlled study of kinderg:arten effects 
during the demonstration year. Four kindergarten teachers introduced PLATO 
to their ^A.M. or P.M. cJLasses in the first semester, and delayed use of the 
terminals until second semester for their other half^-^day class* This made 
possible a first-semester within-teacher comparison of achievement among 68 
PLATO and 67 non-PLATO kindergarteners, ba^inced for momi^ and afternoon 
•(traditionally, less mature childre^j0re tracked into morning kindergarten) 
exposure to PLATO. At the end of the first semester, PLATO children had aver- 
aged only about five, hours (30 sessions) on PLATO, and had encountered 
continuing difficulties with the mechanics of discs, headphones,^ and the touch 
panel, as well as with system failures. Phonics lessons, which had been rede- 
signed over the summer, proved to be confusing, and disrupting requests for ^ 
teacher help were not infrequent. 

It is difficult to envisage the mechanism by which t^is apparent l]^innoc- 
uous interventiorf could have produced a significant effect on individual 
children, who were exposed to PLATO *s ministrations for an average of five 
minutes per day. Yet, the impact on group achievement was a^ignif leant and 
negative^ suggesting that the level of classroom disruption may have been 
gifeater than was apparent to teachers or observers. The results on midyear 
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standardized tests were slgnif icantJLy negative (4, 7 points) for t^e two PLAXb 
A.M* glasses, even taore negative (8*7 points) for one PLATO P,JJ» cla^s^ but 
positive on the average <1.6- points) f or *the other PLATO P.M*^ class ^ reflecting 
a PL^O by pretest interaction^ which 'balanced a negative iffect fo^ initially 

lower-scoring students with a positive effect^^or initially^ ttfore' able students* 

^ * > ' * , ' "'^^ • . • ^ , ) 

PLATO effects on^'the curmculum-specif ic test balanced out* being about -2 *for- 

PLATO A.M. students and abc?uf-^+^ for the (more able) PLATO- P*M. • students. 

^ ^ It thus seemed that the PLATO Elementary ^Reading CarriGul\m^ in its ^> 

first -semester form, had ^ if anyth4.ng,"' a negative effect on ^kindergarten - * 

reading achievement. Although the curriculum was again revised, in the second 

semester, it was not possible to assess tha effect of these final revisions, 

/^Reading attitudes were 'assessed in the pilot .and deinbnistrati^n y'ear^ among 

first-graders. Al,though attitudes toward PLATO were clearly positive among the 

children in both years, there was no evidence that these p6sitive feelings 

transferred to the activity of reading. 
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Findings and Conclusipns 

-4 

Two major findings em^erged from, this evaluation: _ ♦ 

1» The PLAtO Elementary Matherftatics Curriculum, in spit^ or because 
^f its first-draft form and competing teaching philosophies, wa^ a 
clear success when delivered, in an '"add on" mode, and was, particularly 
successful when integrated with teacher mathematics coverage. 

The mathematics treatment was associated with l^ti^e achievement 
gains in grades fourrthrough sin and with moderate positive attitude 
outcomes in grades four and five when it was presenting material that 
was neither overly familiar nor too, far a^ove the students* readiness 

level. The highly structured fractions strand, although sometimes 

... , ' 
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less fun than whole numbeifs or graphs, was particularly effective 
in conveying understanding and skills . . 

A particularly iurportant outcome was revealed in positive 
effects oh ^instruments designed Jto measure students^ understanding 
of and ability to repfesent concepts aad operations, beyond mere , 
facility in manipulation' of symbols. Th^ PLA.TO system here demon- 
^trated tMt it was capable of te^chingj as well as of providing 
drill 5pad practice of concepts already introduced by classroom 
teachers* . , 
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The BLATO -Elementary Keaiiing Curriculum d^onstrated negative 

- impact on first-grade reading. achievement in the pilot year and on 
kindergarten reading readiness achjpcement in the first sem^ter 
of th^ demonstration yeai». No efifect'^on attitudes toward reading 
was found. Additional ancillary hardware (in particular the Audio 
device) with attendeiit production and implementation problems, and 

. the immaturity of the target population (ages five to seven) , were 
factors ,in this failure. H<3wever, in the ot)inion of the evaluators,^ 

f 

the discrete and slow-moving curriculum, which, in contrast to the . 
/ mathematics. lessens, did nqt focus strongly on meaning or under- 

♦ 

Standing, was a major contributing factor to t^is disappointing 

outcome* The reading development grou]^ worked according to an 

a priori hierarchiAl theory of reading acquisition which kept 

• * 

curriculum developroexvt on its initial path long after it became 
clear even to most of the reading developers that the approach was 
not reaching its^goal* 

In addition to these principal findings, we offer five conclusions, which 
although groimded in the experience of this projecf, may generalize beyond 
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this particular demonstration of PLATO: » , > 

I, Teacher effects are real, large, ^and idiosyncratic* Requiring 

that a program demonstrate impact by swamping teacher variance (l.e*, 

^ - tf , . , * 

considering teacher effects to be part of ,the "error term" on the 

grouttids that they are not subject to control and hence are policy- 

Irrelevant) is tantamount to sayiiig that a "treatment" is unitary and 

• . ■ . <' 

that teachers are interchangeable. Neither of theae_aft§mptiibiia-4^^ 
tenaijle, nor is either likei^to lead to progress In educational 
policy ♦ * \. 

•^N 21 In spite of its apparent replicability ^ computer -assisted instruc- 
tion is a treatment which interacts with its setting, arid is ho better 
, thart* the curriculxm it delivers* The PLATO system is relatively ^ 
~ "transparent", ip the sense that it imposes fjew limitations beyond 
those of the quality of lessons, and routing procedures (including 
teacher deci-siotis) implemented on the system, , 

* p-*^ Elementary teachers demand.* and perform more effectively wljen 
given , control over curriculum* The data on individual students 
(particularly false -negative, or '*goofing off dat:|i) are not ^ 
accessibly ,to automated collection, nor do there exiaft "teacher- 

♦ 

proof" algorithms for reducing these dat^ to currlcular decisions* 

The individual trade--offs in foregone alternative activities 
♦ 

("opportunity costs") for students and. teachers were not accessible 
to the. evaluation, but neverthelete were real, and important • 

4, Simultaneous system and curriculum development i« hazardous • 
In general » much more attention should be devoted tou cox|rseware 
than has been the norm in technological innovation* Preference 
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should be Riven to developers who are immersed the subject-matter* 
who have extensive teaching experience, end wh6> ideally, have a tfack 
record of successful curriculum development. In ttje subject-matter ±t\ 
other media. Those who are first attracted to the new ^edium and then 
begin to cast around for something^to teach with it are not likely to ' 
develop effective curricula_at first pa«sr ^ , , 

5* Teachers and students were quite positive about PLATO and its 
potential. We concur, in that the mediym is attractive, flexible, 

highly interactive, and offers immediate feedback to lesson authors. 

■* 

PLAT^O has dpmonstrate^d its potential as a curriculum test bed, for 

*^ ^ 

refinement and perfecting of lesson ideas first tried out in the 
classroom by ''talented curricultmi developers* We would recommend 
support of such use, f Or^eventual .translation to iilore Itoited and 
economical delivery systems* However, without considerable cost 
reduction, particularly in communication «osts, we do not see PLATO IV 
as an economically viable delivery system for elementary schools, 
even with lessons as attractive and effective as those developed by 
the PLATO elementary mathemati<^ groups. 



